The present invention relates to computer graphics. More specifically, the present invention relates to methods and apparatus for creating interactive depth of field effects using simulated heat diffusion.
Depth-of-field (DOF) effects are essential in producing computer graphics imagery that achieves the look and feel of film. Unfortunately, the computations needed to compute these effects have traditionally been very slow and unwieldy. As a consequence, the effects are both costly to create and difficult to direct. Accordingly, accurate computation of depth-of-field effects in computer graphics rendering is generally very time consuming, creating a problematic workflow for film authoring. The computation is particularly challenging because it depends on large-scale spatially-varying filtering that must accurately respect complex boundaries.
Typically, in order to generate a high-accuracy result, a DOF computation must combine information about rays that pass through different parts of a lens. A variety of real-time DOF solutions have been proposed for games, but the compromises in quality required to achieve the necessary frame rates have made these attempts unsuitable for film. Additionally, some approaches to computing DOF vary in the detail with which they model the lens and light transport, their performance-quality tradeoffs, and in their suitability to implementation on graphics hardware.
In one example, an accumulation buffer method takes the approach of simulating DOF effects by blending together the results of multiple renderings, each taken from slightly different viewpoints. Unfortunately, the accumulation buffer method requires a large collection of renderings to achieve a pleasing result, and the enormous geometric complexity of film-quality scenes makes this prohibitive. It is not unusual for the geometry of film-quality scenes to exceed any available RAM, so doing multiple passes through the original geometry is out of the question for interactive film preview.
In order to achieve interactive performance, most film authoring rely largely on two post-processing approaches which can be divided into two major categories: scattering and gathering. Scatter techniques (also known as forward-mapping techniques) iterate through the source color image, computing circle of confusions for each source pixel and splatting its contributions to each destination pixel. Proper compositing requires a sort from back to front, and the blending must be done with high-precision to avoid artifacts. Distributing energy properly in the face of occlusions is also a difficult task. Though scatter techniques are commonly used in non-real-time post-processing packages, they are not the techniques of choice for today's real-time applications, primarily because of the cost of the sort, the lack of high-precision blending on graphics hardware, and the difficulty of conserving total image energy.
Gather techniques (also known as reverse-mapping techniques) do the opposite: they iterate through the destination image, computing the circle of confusion for each destination pixel and with it, gathering information from each source pixel to form the final image. The gather operation is better suited for graphics hardware than scatter. Indeed, the most popular real-time DOF approaches today use this technique. Nonetheless, the gather operation is still not very well matched to today's Single Instruction, Multiple Data (SIMD) graphics hardware, because of the non-uniformity of the sizes of circles of confusion. The gathering method also has difficulty with edge discontinuities and edge bleed.
Moreover, even if efficiently implemented on target hardware, standard gather and scatter techniques have poor asymptotic complexity because the amount of work they do is the product of the number of pixels in the image and the average area of the circle of confusion. For example, For an n×n image, standard gather and scatter algorithms are O(n4), which is clearly problematic for high-resolution film-quality images. In order to bring the computational cost down to a level that permits real-time performance, some implementations compute large blur kernels by down-sampling. While down-sampling may provide a sensible compromise to achieve adequate performance for games on typical hardware, down-sampling causes artifacts that are unacceptable for film authoring, especially film preview. Another problem with down-sampling is that existing techniques do not allow large-scale blurs to be computed efficiently in ways that respect the critical boundaries between in-focus objects and those that are out-of-focus. As a result, the acceleration methods used to down-sample will cause unacceptable color bleeding.
Accordingly, what is desired are improved methods and apparatus for solving the problems discussed above, while reducing the drawbacks discussed above.